Skip to content

Conversation

rankaiyx
Copy link

@rankaiyx rankaiyx commented Aug 2, 2025

Description

This PR adds API key authentication to the server. Now users can secure their endpoints with a simple API key.

Key features:

  • API key is optional (disabled by default)
  • API key is passed via --api-key command line argument (not hardcoded or using environment variables)
  • Authentication follows OpenAI API standard (Bearer token in Authorization header)
  • No breaking changes - existing functionality remains unchanged

How to Test

  1. Start server with API key:

    mlc_llm serve  /somemodel --host 0.0.0.0 --device cuda --mode interactive --api-key "mlc-ai-is-awesome"
  2. Test with correct key (should succeed):

    curl http://localhost:8000/v1/models -H "Authorization: Bearer mlc-ai-is-awesome"
  3. Test with incorrect key (should return 401):

    curl http://localhost:8000/v1/models -H "Authorization: Bearer wrong-key"
  4. Test without key (should return 401):

    curl http://localhost:8000/v1/models
  5. Start server without API key (all requests should work without authentication):

    python -m mlc_llm.serve --model your-model

This change follows the minimal modification principle and keeps the API simple and secure.

Copy link

@JaydenChao101 JaydenChao101 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants